|
Author
|
Topic: That Tears It.
|
stat
Member
|
posted 04-03-2008 05:43 PM
  
Is it at all possible for the APA to send out a survey asking for rough numbers as to how many successfully determined (post facto) polygraph exams say, 100 examiners have had in the last 5 -10 (whatever)years passed----so that when someone runs a much publicized false negative, we'll have some figure to estimate our collective successes. Volunteers must do a little work to do this----some real archive searches. When these damn things (FN's) happen, it just sounds so anecdotal to retort "well I have had gobs and gobs of successes in my career----it's all worth it." Single examiner war stories just sound so stupid. We need figures from warm-blooded participants.rantus verbosis fine' [This message has been edited by stat (edited 04-03-2008).] IP: Logged |
Dan S
Member
|
posted 04-04-2008 01:57 AM
Stat:You may want to contact Dr. Frank Horvath regarding that idea. He is over at DACA and is responsible for the research center.
IP: Logged |
Barry C
Member
|
posted 04-04-2008 05:40 AM
Examiners don't respond to surveys, which makes their results problematic. Just a couple years ago the APA (Frank Horvath) did a survey. (I don't recall if it included this info.) He presented on a portion of it last year, and he's going to talk about more of it this year (at APA), but one of the problems of the survey is that we don't know how well it represents the polygraph community as a whole. Would people who aren't successful all that often contribute to the survey, for example, or would only the best? How many examiners are there outside the APA who wouldn't see the survey and have a chance to respond? In other words, a random sample is tough, and without one, your survey isn't scientific.Having a (good) statistic would be nice, but I can already write the other side's response. You could end up with proof that there are successes, but we already know that - and so do our detractors. They just attribute it to chance. IP: Logged |
rnelson
Member
|
posted 04-04-2008 07:59 AM
Market researchers and political pollsters know that a stratified (non-random) sample can represent a population well, even with smaller samples.They're sure not perfect, but they are sometimes uncomfortably accurate. So, if the APA has demographic descriptives on APA members, such as: geographical region, employment/environment (LE, private, gummit, etc., education, and years of experience (and perhaps some other things, I dunno). They could actually construct a representative stratified sample with as few as 80 to 100 participants. Stratified selection is a cure for the sampling problem of self-selection. Then calculate the descriptives of interest for the stratified sample and make population estimates from that. This is how sociologists achieve reasonable estimates of things things like population impact for nations in crisis (e.g. the number of civilian deaths in Iraq) without a full-scale census. .02
r
------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)
IP: Logged |
Barry C
Member
|
posted 04-04-2008 08:14 AM
That may be true; however, I'm going by my memory of what Frank said at his presentation. He started out being very cautious stating that his survey was helpful, but it was questionable as to whether one could make any generalizations because of the number of responses.My point isn't that it can't be done. It was that examiners don't contribute enough to help themselves. If people would take a few minutes and offer what they know, then we could get somewhere. IP: Logged |
rnelson
Member
|
posted 04-04-2008 08:42 AM
Do disrespect to Dr. Horvath, buy yes, it can be done. It is possible to draw generalizable inferences from small non-random samples. What is necessary is to think like a market researcher and not a scientific researcher. Just stratify the sample according to the known demographic descriptives that are correlated with the statistic of interest. Sure its necessary to be cautious. That is true with all samples. But it is possible to take a self-selected sample and make it more representative by stratifying it. Political pollsters do it all the time, and draw interesting conclusions from surveying 200 people in a region or 2000 people nationally. Actually, they may survey many more people and select their N based on fidelity to their strata. So, OK, I get you. Its not as simple as I make it out to be. It is still possible to achieve some forms of representativeness (is that a word?) from small self-selected sample. Its just more limited than a census, or large random sample. But even large random samples have their limitations in terms of distribution shape and representativeness/bias. Thats why we want several different sample, and construct distributions of sampling distributions. Stratified samples actually work better in many situations. Like when you don't have years to construct those distributions. r
.02
r
------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)
IP: Logged |
stat
Member
|
posted 04-04-2008 08:49 AM
Barry said;
quote---------------------------------That may be true; however, I'm going by my memory of what Frank said at his presentation. He started out being very cautious stating that his survey was helpful, but it was questionable as to whether one could make any generalizations because of the number of responses.
---------------------------------end quote Well, the whole field of survey statistics is just that----generalizations based on a sliver of a given population. It's like saying friend chicken might be greasy. That's the whole point. The surveyed representation will be varied and strange, just like our ranks. again Barry;
-quote----------------------------------
My point isn't that it can't be done.
---------------------------end quote---- Barack Obama says "yes we can." I like that. The trick isn't so much the survey, but as Barry said, getting the volunteers. I would be interested in what kinds of incentives would be considered ethical within the academic rules of survey rendering.
[This message has been edited by stat (edited 04-04-2008).] IP: Logged |
Barry C
Member
|
posted 04-04-2008 08:52 AM
Perhaps that what he's gearing up for this year? I don't know. Last year's presentation was based on an incomplete analysis, which was one reason for his hesitancy.Have the political samplers been right about anything this year? Hillary was supposed to have won a long time ago, and J.M. didn't have a chance.... If "representativeness" isn't a word, we'll have to petition Webster and American Heritage. It works for me. IP: Logged |
Barry C
Member
|
posted 04-04-2008 08:58 AM
quote:
I would be interested in what kinds of incentives would be considered ethical within the academic rules of survey rendering.
It depends on whom you read. If you read the statisticians (purists), they almost never meet a survey sample they like. Social scientists, on the other hand, are much more open (because you need to be practical). I went to one paper presentation where the researcher got data from people on Craig's List. (A site I've still yet to check out.) Stat, You could do this, you know. The anti crowd never has a problem citing a study with which we might question the methodology, but once it's out there, it's out there for all to cite. We could do the same. I'd like to see more: That info along with charts. That way you establish a confirmed case database and the other info to answer some of the other questions. For example, is there a difference between male and female examiners depending on the sex of the examinee? There are lots of questions we could tackle if examiners would just participate.
IP: Logged |
Barry C
Member
|
posted 04-04-2008 09:00 AM
The way to quote (and save some typing is to write (quote) (except use [] instead, but if I do that here, this will look like a quote) at the beginning of the text, and then use (/quote) at the end of the quote.So, (quote) paste text (/quote) and you're done - but remember to replace the () with []. IP: Logged |
stat
Member
|
posted 04-04-2008 09:26 AM
So, we're trying to get some survey numbers regarding field succeses in multi-modality polygraph testing. Rather than all of the talk, perhaps a grocery list of objectives would be helpful---a grocery list is a list of things sans the reason we need them (needs analysis) as needs analysis' tend to bulk and intimidate participants. When I write dwn potatoes on my grocery list, I need not write down why I need them----that's a given. (killer Eric beef stew)1.Pamphlet for Couples written by Therapist 2. Survey of confirmed TP's and TN's of 100. 3. Internet Preferances/ appointed Webmaster for APA.
4. Entertainment Polygraph QC
5. Hot Rod as door prize for people w/ initials E and J. IP: Logged |
rnelson
Member
|
posted 04-04-2008 10:56 AM
quote:
Have the political samplers been right about anything this year? Hillary was supposed to have won a long time ago, and J.M. didn't have a chance....
That's the fun of it all. .02
r
------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)
IP: Logged |
stat
Member
|
posted 04-04-2008 12:48 PM
Political polling and archive-driven polling aren't anywhere close to being the same. Political polling is nebulous and can change faster than the speed of smell, whereas archival polling is immutable, and unchanged by tidal "feelings."IP: Logged |
Barry C
Member
|
posted 04-04-2008 12:53 PM
Easy now stat. It's usually Ray and I who are being too serious.IP: Logged |
Barry C
Member
|
posted 04-04-2008 03:46 PM
I just looked over Dr. Horvath's info from last year, but it was only a partial analysis, so I'm not sure what the answer is. I can ask him what we might be able to glean from the survey he did, but that may be part of this years session. Since he reads these posts occasionally, maybe it just made the list?There is one question that would be interpreted in many different ways depending on whether your a friend or foe: "Based on your experience in administering specific issue polygraph examinations using a 'control-question' procedure, how many errors do you know with certainty that you have made? CQ Errors NDI as DI... 1.1 [Mean] (2.8) [SD]
DI as NDI... 1.5 [Mean] (3.2) [SD]" How many exams? I don't know, but there were 774 male examiners, 95 female, and 7 who apparently weren't sure. I can vaguely recall filling out that survey (online), and I'm pretty sure it asked for an estimate of tests conducted, with breakdowns. So, we're probably talking about a large number of tests.
IP: Logged | |
Polygraph Place Bulletin Board
